An Embedded Marked Point Process Framework for Three-Level Object Population Analysis

In this paper we introduce a probabilistic approach for extracting complex hierarchical object structures from digital images used by various vision applications. The proposed framework extends conventional Marked Point Process (MPP) models by (i) admitting object-subobject ensembles in parent-child relationships and (ii) allowing corresponding objects to form coherent object groups, by a Bayesian segmentation of the population. Different from earlier, highly domain specific attempts on MPP generalization, the proposed model is defined at an abstract level, providing clear interfaces for applications in various domains. We also introduce a global optimization process for the multi-layer framework for finding optimal entity configurations, considering the observed data, prior knowledge, and interactions between the neighboring and the hierarchically related objects. The proposed method is demonstrated in three different application areas: built in area analysis in remotely sensed images, traffic monitoring on airborne and mobile laser scanning (Lidar) data and optical circuit inspection. A new benchmark database is published for the three test cases, and the model's performance is quantitatively evaluated

3D People Surveillance on Range Data Sequences of a Rotating Lidar

In this paper, we propose an approach on real-time 3D people surveillance, with probabilistic foreground modeling, multiple person tracking and on-line re-identification. Our principal aim is to demonstrate the capabilities of a special range sensor, called rotating multi-beam (RMB) Lidar, as a future possible surveillance camera. We present methodological contributions in two key issues. First, we introduce a hybrid 2D--3D method for robust foreground-background classification of the recorded RMB-Lidar point clouds, with eliminating spurious effects resulted by quantification error of the discretized view angle, non-linear position corrections of sensor calibration, and background flickering, in particularly due to motion of vegetation. Second, we propose a real-time method for moving pedestrian detection and tracking in RMB-Lidar sequences of dense surveillance scenarios, with short- and long-term object assignment. We introduce a novel person re-identification algorithm based on solely the Lidar measurements, utilizing in parallel the range and the intensity channels of the sensor, which provide biometric features. Quantitative evaluation is performed on seven outdoor Lidar sequences containing various multi-target scenarios displaying challenging outdoor conditions with low point density and multiple occlusions

A Two-Layer Marked Point Process Framework for Multilevel Object Population Analysis

In this paper we introduce a probabilistic approach for extracting object ensembles from various digital images used by machine vision applications. The proposed framework extends conventional Marked Point Process models by allowing corresponding entities to form coherent object groups, by a Bayesian segmentation of the population. A global optimization process attempts to find the optimal configuration of entities and entity groups, considering the observed data, prior knowledge, and local interactions between the neighboring and semantically related objects. The proposed method is demonstrated in three different application areas: built in area analysis in remotely sensed images, traffic monitoring on airborne Lidar data and optical inspection of printed circuit boards

Color models of shadow detection in video scenes

In this paper we address the problem of appropriate modelling of shadows in color images. While previous works compared the different approaches regarding their model structure, a comparative study of color models has still missed. This paper attacks a continuous need for defining the appropriate color space for this main surveillance problem. We introduce a statistical and parametric shadow model-framework, which can work with different color spaces, and perform a detailed comparision with it. We show experimental results regarding the following questions: (1) What is the gain of using color images instead of grayscale ones? (2) What is the gain of using uncorrelated spaces instead of the standard RGB? (3) Chrominance (illumination invariant), luminance, or ”mixed” spaces are more effective? (4) In which scenes are the differences significant? We qualified the metrics both in color based clustering of the individual pixels and in the case of Bayesian foreground-background-shadow segmentation. Experimental results on real-life videos show that CIE L*u*v* color space is the most efficient

Reconstruction of 3D Urban Scenes Using a Moving Lidar Sensor

In this report, we propose algorithms which interpret and display 3D environments.The input of this procedure is a LiDAR sensor mounted atop of a car. The sensor outputs a data stream covering more than 100 meters radius of space, collecting data at 15Hz. The recording is done in real environment on the streets of Budapest in real time, while the processing is offline, implemented on CPU keeping in mind the future implementation on GPUs to reach real time data processing. The aim is to segment several region classes (such as roads, building walls, vegetation) and to identify specified objects (such as people, vehicles, traffic signs) in the point clouds through a presegmentation step. To achieve this classification, we need several features such as the color and geometrical properties of the specified objects and their possible geometrical and physical interactions. Also, we need to take into account the time domain features calculated based on the LiDAR data stream. After this presegmentation step we are able to reconstruct building facades in 3D and to track the detected objects in the 3D space. Also, in the future, this processed data set can be registered against 2D images provided by conventional cameras to reproduce realistic, colored 3D virtua

Markovian framework for structural change detection with application on detecting built-in changes in airborne images

In the paper we address the problem of change detection in airborne image pairs taken with significant time difference. In reconnaissance and exploration tasks, finding the slowly changing areas through a long tract of time is disturbed by the temporal parameter changes of the considered clusters. We introduce a new joint segmentation model, containing two layers corresponding to the same area of different far times and the detected change map. We tested this co-segmentation model considering two clusters on the photos: built-in and natural/cultivated areas. We propose a Bayesian segmentation framework which exploits not only the noisy class-descriptors in the independent images, but also creates links between the segmentation of the two pictures, ensuring to get smooth connected regions in the segmented images, and also in the change mask. The domain dependent part of the model is separated, therefore the proposed structure can be used for significantly different descriptors and problems also